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Introduction 


Managing  uncertainty  in  complex  domains  continues  to  remain  a  difficult  task 
especially  during  knowledge  acquisition  and  verification  auid  validation.  There 
are  a  wide  variety  of  approaches  from  fuzzy  logics  to  probabilistic  networks 
[6,  15,  10,  7,  13,  4,  12,  11,  5,  1].  The  difficulty  lies  in  creating  a  knowledge 
representation  with  the  right  blend  of  ftexibUiiy  and  sound  semaniics.  For  the 
human  expert  and  knowledge  engineer,  flexibility  and  intuitiveness  eases  the 
acquisition  and  organization  of  knowledge  for  the  target  domain.  On  the  other 
hand,  sound  and  formal  semantics  prevents  confusion  concerning  the  interaction 
between  the  different  sources  of  uncert2dnty.  This  serves  to  avoid  anomalous 
reasoning  behaviors  and  guarantees  internal  consistency. 

Many  other  pragmatic  concerns  also  drive  our  choice  of  knowledge  repre¬ 
sentations  such  as  accessibility  and  incompleteness.  By  accessibility,  we  mean 
whether  the  information  required  by  the  knowledge  representation  (in  addition 
to  that  of  the  target  domain)  is  actually  available.  For  example,  in  a  Bayesian 
network  [7],  certain  conditional  probabilities  may  not  exist  or  are  not  meaning¬ 
ful  in  the  target  domain.  Accessibility  immediately  leads  to  completeness.  In 
the  real  world,  having  complete  information  is  typically  unattainable,  thus  a 
representation  must  have  the  ability  to  handle  and  recognize  incompleteness  as 
it  occurs. 

Most  agree  that  encoding  knowledge  in  terms  of  logical  *^if-then  style  rules  is 
the  simplest  and  most  intuitive  approach  to  organization  [3].  While  uncertainty 
can  be  explicitly  modeled  as  various  exceptions  to  the  rules,  the  number  of 
such  exception  rules  quickly  explodes.  On  the  other  hand,  probability  theory 
is  and  has  been  an  accepted  language  both  for  the  description  of  uncertainty 
and  for  making  inferences  from  incomplete  knowledge.  However,  the  general 
language  of  probabilities  is  too  unconstrained  maJcing  it  a  poor  mechanism  for 
easily  organizing  and  manipulating  information.  Without  additional  knowledge 
such  as  independence  conditions,  the  various  sources  of  uncertainty  can  not  be 
resolved  or  combined. 

In  this  paper,  we  present  a  new  model  for  knowledge  representation  under 
uncertainty  called  Bayesian  Knowledge-Bases  (abbrev.  BKBs^.  BKBs  marry  the 
strong  formalisms  for  probability  theory  with  an  ^if-then”  rule  structure.  We 
will  prove  that  BKBs  provide  a  sound  and  consistent  representation  of  knowledge 
and  that  the  results  they  generate  will  not  be  inconsistent  even  in  the  face  of 
incompleteness-  Yet,  even  with  all  this  formalism,  BKBs  still  retain  the  flexibility 
of  “if-then”  style  rules  by  providing  intuitive  probabilistic  semantics. 

Section  2  presents  the  basic  foundations,  models  and  properties  necessary  to 
define  BKBs.  Next,  in  Section  3,  the  semantics  of  BKBs  are  precisely  defined  and 
then  proven  to  be  consistent  with  probabilistic  theory.  Findly,  in  Section  4,  we 
summarize  our  results  and  present  our  conclusions. 
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Foundations 


We  define  a  knowledge-bast  to  consist  of  two  components,  a  knowledge-graph  and 
an  inferencing  scheme.  The  knowledge-graph  represents  objects/world  states 
amd  the  relationships  between  them.  The  inferencing  scheme  details  how  to 
manipulate  or  interpret  the  information  in  the  knowledge-graph.  The  latter  can 
be  represented  as  subgraphs  of  the  knowledge-graph. 

Notation,  fft  denotes  the  real  numbers,  S?"*"  denotes  the  non-negative  reals,  and 
$  denotes  the  empty  set. 

We  define  our  knowledge-graph  as  follows: 

Definition  2.1.  A  correlation-graph  G  =  (/U5,  jE?)  is  a  directed  graph  such  that 
7  n  5  =  $  and  E  C  {I  x  S}  U  {S  x  /}.  Furthermore^  for  all  a  €  5,  (a,  6)  and 
(a,  6')  are  tn  E  if  and  only  ifb^  6'.  {/  U  5}  are  the  nodes  of  G  and  E  are  the 
edges  of  G.  A  node  in  I  is  called  an  instantiation-node  (abbrev.  I-node)  and  a 
node  tn  S  ts  called  a  support- node  (abbrev.  S-node). 

I-nodes  represent  the  various  states  of  the  world  such  as  the  truth  or  falsity  of 
a  proposition,  S-nodes,  on  the  other  hand,  explicitly  embody  the  relationships 
between  the  I-nodes.  See  Figure  2.1. 

Notation.  Let  a  be  any  node  in  7  U  5.  =  {6|(6,  a)  G  are  the  immediate 

predecessors  of  a  in  graph  G. 

Let  X  be  a  partition  on  7,  Intuitive,  will  denote  the  groups  of  I-nodes 
(states)  which  are  mutually  exclusive.  For  example,  this  can  be  used  to  represent 
random  variables  with  discrete  but  multiple  instantiations. 

Notation.  |  •  [  denotes  cardinality. 

Definition  2.2.  G  is  said  to  I-respect  ir  if  for  all  cells  <t  in  tt,  for  any  S-node 
b  £  Sj  \crr\  {Df  U  {a})|  <  1  whenever  (6,  a)  G  E. 

Basically,  mutually  exclusive  I-nodes  cannot  be  directly  related  to  each  other 
through  the  S-nodes.  Next,  we  define  mutual  exclusion  between  S-nodes. 

Definition  2.3.  Two  S-nodes  bi  and  62  in  S  are  said  to  be  mutually  exclusive 
with  respect  to  t  if  there  exists  two  distinct  I-nodes  cx  and  C2  in  some  cell  a  of 
TT  such  that  Cl  G  7>^  and  C2  €  7)^. 

Definition  2.4.  G  is  said  to  S-respect  tt  if  for  all  I-nodes  a  in  7,  any  two  distinct 
S-nodes  bi  and  62  i«  nrc  mutually  exclusive. 

Definition  2.5.  G  is  said  to  respect  tt  if  G  both  I-respects  and  S-respecis  tt. 

To  complete  our  knowledge-graph,  we  define  a  function  w  from  5  to  St.  This 
serves  as  the  mechanism  for  handling  uncertainty  in  the  relationships. 

Definition  2.6.  A  knowledge-graph  K  is  a  S-tuple  (G,  t£;,x)  where 

•  G  =  (7  U  5,  jE*)  is  a  correlation- graph 

•  w  is  a  function  from  S  to  U  {00}.  For  each  a  G  5,  u;(a)  is  the  weight 
of  a. 
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Fig.  2.2.  Example  of  a  knowledge  graph.  Let  x  consist  of  cells  {oO}, 
{al,  a2},  and  {am}. 


•  ir  is  a  partition  on  I. 
and  G  respects  x. 

We  now  formally  define  our  second  component,  the  inferencing  scheme.  We 
begin  by  defining  basic  properties  necessary  to  inferencing  over  our  knowledge- 
graphs.  Such  properties  include  preserving  the  relationships  between  the  objects 
in  our  graph,  etc. 

Let  r  =  (/'US',  E*)  be  some  subgraph  of  our  correlation-graph  G  =  (7U5,  E) 
where  /'  C  /,  5'  C  5,  amd  E'  C  E.  Furthermore,  r  has  a  weight  U7(r)  defined  as 
follows: 

*65' 

Definition  2.7.  An  I-node  a  E  T  is  said  to  be  well-supported  tn  r  if  there 
exists  an  edge  (6,  a)  in  E' .  Furthermore,  r  is  said  to  be  well-supported  if  for  all 
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I-nodes  a  in  a  is  well-supported. 

Each  I-node  must  have  an  incoming  S-node  in  r. 

Definition  2.8.  An  S-node  b  £  is  said  to  be  well-founded  in  r  if  for  all 
(a,  6)  E  E,  (a,  6)  G  E* .  Furthermore^  r  is  said  to  be  well-founded  if  for  all 
S-nodes  b  in  S',  6  is  well-founded. 

If  an  S-node  6  is  present  in  r,  then  all  incoming  I-nodes  (conditions)  to  6  in 
G  must  also  be  present  in  r. 

Definttion  2.9.  An  S-node  i  E  S'  is  said  to  be  well-defined  in  r  if  there  exists 
an  edge  (6,  a)  E  Furthermorej  r  is  said  to  be  well-defined  if  for  all  S-nodes 
6  in  S',  6  is  well-defined. 

Each  S-node  in  r  must  support  some  I-node  in  r. 

Definition  2.10.  r  is  said  to  be  an  inference  over  K  if  r  is 

•  well-supportedf 

•  well-founded, 

•  well-defined, 

•  acyclic,  and 

•  For  all  cells  a  in  ir,  |/'  H  o’]  <  1. 

Furthermore,  r  is  said  to  be  a  complete  inference  over  K  if 

•  For  all  cells  c  in  ir,  |/'  D  cr]  =  1. 

Given  the  knowledge  graph  in  Figure  2.2,  one  possible  inference  can  be 
seen  in  Figure  2.3.  An  inference  scheme  for  our  knowledge-base  will  consist  of 
inferences  over  our  knowledge-graph. 

Definition  2.11.  A  knowledge-base  fC  is  an  ordered  pair  {K,R)  where  K  = 
(G,  tz;,  tt)  is  a  knowledge-graph  and  R  is  a  non-empty  collection  of  inferences 
over  K  called  the  inference  graphs. 

With  respect  to  our  knowledge-base,  we  can  derive  properties  for  inference 
graphs  which  ao'e  especially  pertinent  to  knowledge-bases. 

Theorem  2.1.  Let  r  =  (/'  U  S',F^')  E  H.  For  each  I-node  a  E  there  exists  a 
unique  S-node  b  £  S'  such  that  {b,a)  £  E'. 

Proof  Since  r  is  well-supported,  there  exists  b  £  S'  such  that  (6,  a)  E  E'. 
Now,  assume  b'  £  S'  and  (6', a)  E  E'  but  b^V.  Since  G  S-respects  tt,  b  and 
b'  must  be  mutually  exclusive.  This  implies  there  exists  two  distinct  I-nodes  c 
and  c'  in  some  cell  <t  in  it  such  that  c  E  Df  and  c'  E  Df,.  Thus,  c  and  o'  aire 
both  in  I'.  However,  this  violates  Definition  2.10.  Contradiction.  □ 

Theorem  2.1  simply  states  that  each  I-node  in  an  inference  graph  is  sup¬ 
ported  by  one  and  only  one  S-node. 

Proposition  2.2.  |/'l  =  |S'|. 

S-nodes  not  only  serve  as  our  mechanisms  for  representing  uncertainty  but 
also  serves  to  ‘‘support”  or  “explain”  the  I-node  it  dominates.  Since  the  I-nodes 
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Fig.  2.3.  Example  of  an  inference.  Let  x  consist  of  cells  {aO},  {al,a2}, 
and  {am}. 


are  used  to  describe  the  various  states  of  the  world,  ambiguity  implies  that  there 
may  be  multiple  explanations  for  a  single  state. 

Let  ri  =  (/i  U  5i,  jS!)  and  r2  =  {h  U  -5*2,  £^2)  he  inference  graphs  over  K, 

Theorem  2.3.  The  following  are  equivalent: 

•  /i  C  J2. 

•  5i  C  S2 . 

•  El  C  £2. 

•  ri  is  a  subgraph  of  r2- 

Proof 

Claim:  7i  C  I2  implies  5i  C  S2. 

Assume  Ii  C  I2  but  Si  is  not  a  subset  of  S2.  This  implies  there  exists  some 
b  £  Si  but  not  in  S2.  Since  ri  is  well-defined,  there  exists  some  a  6  /i  such 
that  (6, a)  G  Ex.  Thus,  a  £  I2.  Since  r2  is  well-supported,  there  exists  some 
fc'  £  S2  such  that  (6',  a)  £  £2.  6‘  ^  6  and  b  must  be  mutually  exclusive  to  t'. 
This  implies  there  exists  some  c  and  c'  in  some  cell  <t  of  ^  such  that  c  £  Df 
and  c'  £  Df,.  Since  both  ri  and  r2  are  well-founded,  c  £  Ii  and  €  /2-  Thus, 
c  and  c'  are  both  in  72-  Contradiction. 

Claim:  Si  C  S2  implies  £1  C  £2. 

Assume  Si  C  S2  but  £1  is  not  a  subset  of  £2.  This  implies  there  exists 
some  (a.t)  £  Ex  but  not  in  £2.  If  a  €  Si,  then  a  £  S2.  By  Definition  2.1,  6 
unique.  Since  r2  is  well-defined,  (a,  6)  €  £2-  Thus,  a  must  be  in  7i.  Since  r2 
well-founded  and  6  G  S2,  (a,i)  must  also  be  in  £2.  Contradiction. 

Claim:  £1  C  £2  implies  ri  is  a  subgraph  of  r2. 
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Assume  E\  C  but  ri  is  not  a  subgraph  of  rj.  This  implies  there  exists 
some  a  6  /i  U  Si  but  not  in  /2  U  S2.  If  a  €  A ,  then  since  ri  is  well-supported, 
there  exists  some  6  6  Si  such  that  (6, a)  €  Ei.  Thus,  a  must  be  in  Si.  Since  ri 
is  well-defined,  there  exists  some  6  €  A  such  that  (a,  6)  €  Ei-  Contradiction. 
Clearly,  if  ri  is  a  subgraph  of  r2,  then  A  Q  A-  D 

Theorem  2.3  provides  another  level  of  consistency  in  that  inferences  over 
portions  of  the  shared  subgraphs  are  identical. 

Corollary  2.4.  The  following  are  equivalent: 

•  A  =  A- 

•  Si  =  S2. 

•  El  =  J?2* 

•  ri  =  r2. 

Corollary  2.4  above  says  that  each  state  of  the  world  is  uniquely  determined 
by  an  inference  graph  in  /C-  Hence,  K  avoids  ambiguity. 

With  respect  to  uncertednty,  we  have  the  following: 

Proposition  2.5.  If  Si  =  S2,  then  w(ri)  =  u;(r2). 

Key  Theorem  2.1.  //A  =  A^  Ihen  w(ri)  =  a;(r2). 

Proof  This  follows  from  Corollary  2.4  and  Proposition  2.5.  □ 

Key  Theorem  2.1  lays  the  foundations  for  our  Bayesian  Knowledge-Base  as 
we  shall  see  later. 

Definition  2.12.  Lei  rj  and  be  two  subgraphs  ofG.  ri  is  said  to  be  compatible 
with  r2  if  there  does  not  exist  two  distinct  Enodes  ai  and  02  m  some  cell  cr  in 
TT  such  that  ai  €  A  02  €  A*  Otherwise,  ri  is  said  to  be  incompatible  with 

This  leads  to  the  following  properties  on  constructing  inferences.  Let  r  = 

(/'uS',F)6i?. 

Lemma  2.6.  Let  h  =  (AciS/i,  Eh)  be  a  subgraph  ofG  such  that  r  is  a  subgraph  of 
h  and  for  all  a  E  h,  there  exists  at  most  one  edge  (6,  a)  €  Eh  for  some  6  G  S/j. 
For  each  a  £  Ih  —  T ,  a  cannot  be  an  ancestor  in  h  of  any  Fnode  a'  G 

Proof  Assume  there  exists  an  a  G  A  ~  such  that  a  is  eui  ancestor  of  some 
I-node  a!  .  Without  loss  of  generality,  assume  (a,  6)  and  (6,  a')  are  in  Eh  for 
some  S-node  6  G  5^.  Since  r  is  well-founded,  this  implies  that  if  6  G  5',  then 
a  G  Hence,  6  is  not  in  5'.  Since  r  is  well-supported,  there  exists  V  G  such 
that  (6', a')  G  E* .  Thus,  both  (6, o')  and  (6', a')  are  in  Eh^  Contradiction.  □ 
From  Lemma  2.6,  we  can  conclude  that  inferences  can  be  constructed  in  a 
bottom-up  manner  from  other  “smaller”  subgraphs  until  we  finish  at  a  complete 
inference. 

Definition  2.13.  rj  is  said  to  be  an  immediate  parent  of  r2  if  ri  is  a  proper 
subgraph  of  r2  and  |A  “  AI  =  1-^2  ~  5’il  =  1- 
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Key  Theorem  2.2.  If  ri  w  a  subgraph  of  r^f  then  there  exists  a  sequence  of 
inferences  such  that  ri  is  the  immediate  parent  of  hi,  hi  is  the 

immediate  parent  ofhi^i  for  i  from  1  to  n  —  1,  and  hn  is  the  immediate  parent 
ofr2. 

Proof  Ftom  Definition  2.13  and  Proposition  2,2,  it  follows  that  n  =  |/2  —  Ii\- 
Since  ri  is  a  subgraph  of  r2,  from  Lemma  2.6,  for  any  a  G  /2  *“  ^  cannot  be  an 

ancestor  of  any  node  in  /i.  Since  all  inferences  aire  acyclic,  there  must  exist  some 
I-node  a  £  I2-  h  such  that  Dl^  C  Ii  for  some  (6,  a)  G  £‘2-  We  can  construct 

hi  =  (/fc,  U  Sht.Es^)  from  ri  as  follows:  Zfc,  =  A  U  {o},  Sji,  =  Si  U  {6}, 

and  Eh,  =  Ei  V  {(6,a)}UeeD’'*i(‘^>^)}- 

inference  over  K  and  hi  is  bl  subgraph  of  r2.  We  can  construct  the  remaining 
hiS  recursively.  D 

Key  Theorem  2.2  further  states  that  such  a  construction  can  occur  as  a 
sequence  of  inferences. 

Definition  2,14.  An  I-node  a  £  I  is  said  to  be  a  root  of  K  if  there  exists  a 

S-node  6  G  5  such  that  (6,  a)  G  E  and  Df  =  $.  l^c  denote  the  set  of  all  roots 

ofKbyV{K). 

For  each  root  a  G  y{K),  the  correspondm^i  root-subgraph  for  a  is  consists 
solely  of  a,  its  associated  S-node,  and  the  edge  between  them. 

Clearly,  all  root-subgraphs  are  inferences  over  K. 

Theorem  2.7.  If  a  €  V{K),  there  exists  a  unique  S-node  b  £  S  such  that 
(6,a)G£. 

Proof  This  follows  from  the  fact  that  Df  =  $  and  that  G  S-respects  x.  □ 
Proposition  2.8.  All  inferences  contain  one  or  more  root-subgraphs. 

Combining  Proposition  2.8  with  Key  Theorem  2.2,  all  possible  inferences 
can  be  constructed  from  the  root-subgraphs. 

Theorem  2.9,  If  ri  is  compatible  with  r2,  then  the  subgraph  r  =  {{Ii  n  I2}  U 
{Si  n  52},  £1  n  £2)  inference  over  K. 

Proof  Clearly,  r  is  acyclic  amd  for  all  cells  <t  in  x,  |(7i  fl  I2)  H  <r|  <  1, 
Furthermore,  r  can  be  easily  shown  to  be  well-founded  and  well-defined.  Finally, 
based  on  Theorem  2.1,  r  can  be  shown  to  be  well-supported.  □ 

Given  two  compatible  inferences,  there  is  a  common  “core”  between  them 
which  is  also  an  inference  and  from  which  they  may  be  constructed  from. 

Next,  consider  the  following  properties  concerning  the  weights  of  the  infer¬ 
ences. 

Notation.  Let  be  a  cell  in  x.  span^  =  Uae<y  d^iiotes  all  the  S-nodes  which 
support  the  I-nodes  in  a. 


9 


Definition  2.15.  K  is  locally  normalized  if  for  each  cell  a  in  r, 


b€S 

whenever  S  C  span^  and  no  two  S-nodes  in  S  are  mutually  exclusive. 

Let  H  =  {h\yh2,  •  -ihn}  where  n  >  1  be  a  set  of  mutually  incompatible 
inferences  over  K  where  A,-  —  {U  U  S,-,  JF*)  for  t  =  1, . . n.  Let  C  =  nJLi/j. 

Key  Theorem  2.3.  If  K  is  locally  normalized,  then  for  any  non-empty  collection 
of  mutually  incompatible  inferences  H  over  K, 

<  1. 

h^H 


Proof  We  prove  this  by  induction  on  the  number  of  S-nodes  in  a  given 
knowledge-graph.  If  a  knowledge-graph  has  exactly  one  S-node  and  is  locally 
normalized,  then  it  is  trivial  to  show  that  the  knowledge-graph  itself  satisfies 
the  above  property. 

Assume  that  the  theorem  is  true  for  any  locally  normalized  knowledge-graph 
of  up  to  n  -*  1  S-nodes. 

Let  X  be  a  locally  normalized  knowledge-graph  with  exactly  n  S-nodes. 
Construct  knowledge-graph  where  =  (P  U5“,£^), 

7-  =  5“  =  Si,E^  =  ULi  Ei,  w^{b)  =  w{b)  for  all  6  E  5",  and 

tt"  =  {a  n  P\<T  £  x}.  Clearly,  is  locally  normalized.  If  is  a  proper 
subgraph  of  K,  then  has  less  S-nodes  than  K  and  we  are  done. 

Assume  the  =  K.  From  Proposition  2.8,  V{K)  ^  Let  a  E  V'(K). 
From  Theorem  2.7,  let  b  be  the  unique  S-node  in  S  such  that  (6,  a)  E  E, 

Let  (7  =  {a  =  00,01,02, ...  jUm}  be  the  cell  in  ir  containing  o.  If  a  only 
contains  a  and  since  o  E  V{K)y  we  can  partition  H  into  two  sets  Ha  and 
H  —  Ha  where  Ha  are  the  inferences  which  contain  a.  Thus, 


h^H 


= 

+  Y 

hiH-H, 

hSH. 

= 

— 

= 

Y 

hen. 

< 

Y 

hiH-H, 
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where  Sh  are  the  S- nodes  corresponding  to  h.  Construct  a  new  knowledge-graph 
K'  from  K  be  removing  nodes  a  and  6.  Similarly,  construct  H*  =  {hi, 
from  H  by  eliminating  a  and  6  from  the  inferences.  Clearly,  the  inference  in 
H  —  Ha  not  affected.  Let  be  the  corresponding  inferences  found  in  Ha- 
Thus, 


^  g-w(h) 


Y 

fc€ir; 

Y  + 

Y 

where  5J^  are  the  S-nodes  corresponding  to  h! .  Since  K*  has  n  —  1  S-node, 

E 

It  follows  then, 

1  =  1 

Otherwise,  if  m  >  0,  then  partition  H  into  three  disjoint  subsets  Hx^Hq,  Hi 
where 

•  hi  e  Hx  if  and  only  if  n  a  = 

•  hi  E  Hq  if  and  only  if  oq  €  li- 

•  hi  E  Hi  if  and  only  if  {oi, . . . ,  am}  H  /,•  ^ 

Since  oq  E  y{H),  let  bo  be  its  unique  associated  S-node.  Let  B  be  the  set 
of  S-nodes  found  in  Hi  that  support  the  remaining  I-nodes  {oi, . . . ,  0^}. 

Now,  construct  a  new  K*  from  K  as  follows: 

1.  Introduce  new  I-node  oj  and  new  S-node  60  for  oj.  Let  w{bo)  =  —  In  [l  —  . 

2.  Replace  cell  cr  by  <ri  =  {ao,  oo}  and  =  {oi, . . . ,  a^}.  _ 

3.  For  each  b  £  introduce  (oq,  6)  and  change  w{b)  to  tz;(6)  —  u;(6o). 

From  our  construction,  K*  is  also  a  locally  normalized  knowledge-base.  See 
Figures  2.4  and  2.5  for  an  example  transformation. 

Next,  construct  H*  from  H  as  follows: 

•  For  all  h  £  HxU  Hot  h  is  still  a  vali^inference  over 

•  For  all  h  E  ^1,  introduce  Oo  and  60  as  well  as  the  additional  arcs  and 
changes  in  weights  outlined  in  the  construction  of  K'  above. 


e"W+  Y,  1. 
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For  each  hi  ^  Hy  let  6  H*  be  the  corresponding  inference  auad  HgyHQy  Hi  be 
the  corresponding  partitions. 

Clearly,  w(/i,)  =  and 

t=l  i=:l 


Since  w{bo)  is  shared  by  all  inferences  in  Hq, 


-  ^-w\ho)^-u,\h)+w\bo) 

_  g-u>'(»o)  g-w'(/i)+ti;'(6o) 

A€HJ 

Similarly,  since  ti;'(6o)  is  shared  by  all  inferences  in  iTj, 


Y 

hen[ 

A€KJ 

h€Hi 


•  Case  1.  e 


u;'(A)+ti;'(to)  >  g-u/'(/i)+u;'(6o) 


hen;  Aenj  hetfj 

<  ^  e-<^\h)  4  g-u,'{»o)  ^  g-<.''(/>)+u-'(»o)  4  ^  g-<./'(h)+tt;'(»o) 


fceHi 


heffi 


=  E E 


,-w'(;»)+u;\6o) 


A€iy4 


A€^i 


since  =  1. 

Construct  a  new  knowledge-base  K"  from  K*  by  eliminating  all  I-nodes 
{oo, ai,a2, . .  .,am}  and  their  attendant  support  nodes  from  K\  Fur¬ 
thermore,  remove  any  S-nodes  which  are  immediate  descendants  of  the 
I-nodes.  If  a  situation  occurs  where  an  I-node  has  no  support  nodes, 
also  eliminate  that  I-node.  Repeat  until  no  I-nodes  need  to  be  removed. 
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Next,  let  u;"(6o)  =  0.  The  inferences  in  and  do  not  need  to  be 
changed.  In  fact,  for  any  h  G  weight  under  if"  is  precisely 

uj”{h)  =  uf'{h)  —  it;'(6o)-  K**  is  also  locally  normalized. 

By  eliminating  the  I-nodes  and  S-nodes,  /f"  has  fewer  than  n  S-nodes. 
By  induction, 

iieHi 

Thus, 

^  g-U.(fc)  <  1 

•  Case  2. 

Similar  to  Case  1,  we  construct  a  new  knowledge-base  by  eliminating 
I-nodes  {ao,ao}.  Again,  K*'  is  locally  normalized  and  has  fewer  than  n  — 1 
S-nodes. 

D 

Key  Theorem  2.3  states  that  local  normalization  implies  global  normaliza¬ 
tion.  This  property  will  play  a  key  role  in  our  semantics  for  these  knowledge¬ 
bases.  In  addition,  this  leads  to  the  following  even  stronger  result  concerning 
local  normalization. 

Key  Theorem  2.4.  Lei  H  be  a  set  of  mutually  incompatible  inferences  over  K 
suck  that  for  each  h  in  H ,  h  is  a  supergraph  of  r.  If  K  is  locally  normalized, 
then 

heH 


Proof  Let  H  =  {hi,  h2j  •  •  >  h„}  and  C"  be  the  subset  of  UjLj/i  such  that 
if  a  G  C',  then  {ca  -  {a})  Pi  7,  =  ^  for  t  =  1, . . . ,  n  where  Ca  is  the  cell  in 
TT  containing  a.  For  each  h,,  w(h,)  =  a;(r)  -{-  for  some  >  0  since  r  is  a 
subgraph  of  hi  and  from  Theorem  2.9. 

From  Key  Theorem  2.3, 


<  I 

»  =  1 


Thus,  it  follows  that 


n 


Construct  a  new  knowledge-base  k!  from  K  by  eliminating  all  I-nodes  from  K 
not  in  U”_i7,-  as  well  as  any  S-nodes  which  are  immediate  parents  or  immediate 
children  of  these  I-nodes.  Furthermore,  eliminate  all  S-nodes  which  do  not  occur 
in  U-Li5i.  For  all  remaining  S-nodes  which  support  the  I-nodes  in  C',  change 
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their  weights  to  0.  Clearly,  jC'  is  a  loczilly  normalized  knowledge-base.  FVom 
Key  Theorem  2.3, 

1=1 

and  consequently 

t  =  l 

Therefore, 

g-u;(h)  ^ 

h^H 

D 

3  Semantics 

In  this  section,  we  describe  how  knowledge-bases  maybe  used  to  represent  prob¬ 
abilistic  information. 

Let  A  —  {Ai,A2,  . .  .,^n}  be  a  set  of  discrete  random  variables^  and  let 
R{Ai)  =  {aj^i,  ai^2^  •  •  • ,  represent  the  set  of  possible  instantiations  to  r.v. 

Ai  for  i  =  1, . . . ,  n. 

Definition  3.1.  An  instantiation  is  an  ordered  pair  (A,  a)  where  A  £  A  and 
a  6  A(A).  (An  insianiiaiion  (A,  a)  ts  also  denoted  by  A  =  a.)  A  collection 
of  instantiations  X  is  called  an  instantiation-set  iff  {A ^  a),  {A,  a*)  in  X  implies 
a  =  a'. 

An  instantiation  represents  the  event  when  a  r.v.  takes  on  a  value  from  its 
range.  Given  am  instantiation-set,  we  can  define  the  notion  of  the  span  of  an 
instamtiation-set. 

Definition  3.2,  Given  an  insiantiaiion’Sei  X ,  we  define  the  span  ofX,  span(A), 
to  be  the  collection  of  r.v. s  in  the  first  coordinate  of  the  instantiations.  Further¬ 
more,  an  instantiation-set  X  is  said  to  be  complete  if  and  only  :/span(A)  =  A- 

The  span  of  an  instantiation-set  simply  denotes  the  r.v,s  which  have  been 
instantiated. 

Let  K  =  (K,  R)  with  K  =  (G,  ly,  tt)  be  a  locally  normalized  knowledge-base 
where  le  consists  of  cells  {tri,  <T2,  . . . ,  such  that  there  exists  a  one-to-one  and 
onto  mapping  functions  :  R\Ai)  — ►  <7,*  amd  Rf{Ai)  C  i?(Ai)  for  i  =  1, . . . ,  n. 

Let  r  =  (/'  U  S\E^)  be  an  inference  over  K.  According  to  Definition  2.10, 
there  exists  at  most  one  I-node  from  each  in  r.  Hence,  we  can  construct  an 
instantiation  set  Xr  from  r  as  follows:  (A,  ,  Ot  j)  €  Xr  if  and  only  if  /i(a,  j)  € 
Intuitively,  if  I-node  /»(oij)  €  this  implies  that  r.v.  A,-  =  aij. 

^Random  variables  will  be  abbreviated  r.v. 
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Proposition  3,1.  r  is  complete  if  and  only  ifXr  is  complete. 

Theorem  3.2.  For  each  instantiation-sei  X,  there  corresponds  at  most  one 
inference  rx  over  K. 

Proof  This  follows  from  Theorem  2.3  and  Corollary  2.4.  0 
From  Theorem  3.2  and  Key  Theorem  2.1,  we  can  associate  a  unique  weight 
w(r)  to  each  Xr^  Our  goal  now  is  to  interpret  this  weights  probabilistically. 

Since  each  u;(r)  is  guaranteed  to  be  non-negative,  0  <  <  1.  In  addi¬ 

tion,  from  Key  Theorem  2.3,  given  any  set  of  mutually  incompatible  inference 
7i  =  {hi, hi, .  ..,hi}, 

«  =  1 

Thus,  given  an  inference  r  and  the  corresponding  instantiation-set  we  can 
interpret  c”*^^**)  as  the  joint  probability  for  Xr- 

Now,  consider  the  S-nodes  and  their  weights  w.  Let  b  £  S  and  (6, 6i)  £  E. 
Note,  6i  is  unique  to  6  from  Definition  2.1. 

Since  G  S-respects  ir,  we  can  associate  a  unique  conditional  probability  with 
each  S-node.  Let  Df  =  •  • -jt*}-  Without  loss  of  generality,  aissume 

bi  G  (Ti  for  t  =  1, . . . ,  fc.  For  6,  associate  the  conditional  probability 

P{Ai  =  aijy\A2  =  a2ja, . . .,  Ajb  = 

where  6^  =  for  i  =  Furthermore,  we  assume  the  following 

conditional  independence  for  the  conditional  probabilities  associated  with  each 
S-node: 


P{Ai  —  aij^\A2  —  02^*3, . . . ,  Ai;  —  = 

P(^Ai  1^2  —  •  j  jk)  (1) 

for  any  instantiation-set  C  such  that  span(C)  n  {Ai,  ...,>!*}  =  $. 

Returning  to  inference  r,  without  loss  of  gener^ity,  assume  P  =  {ci,  C2, . . . ,  Cm) 
and  Cj  G  for  :  =  1, . . . ,  m.  The  joint  probability  P{Xr)  is 

P{Ai  =  aij^\A2  =  a2j3,*.M^t  = 

where  c,-  =  /i(aj  j,)  for  t  =  1, . . . ,  m.  From  Definition  2.7  and  Theorem  2,1,  each 
c,  is  supported  by  a  single  S-node,  say  d,. 

Since  r  is  acyclic,  this  induces  a  topological  ordering  on  the  I-nodes  in  /', 
Without  loss  of  generality,  assume  {ci, . . . ,  Cm)  is  the  ordering  and  Ci  is  not  a 
descendant  of  Cj  for  i  >  Through  the  chain  rule,  we  can  rewrite  P{Xr)  as 

m 

'P(Xr)  —  n ~  j  •  •  • )  (2) 

i-l 
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According  to  our  S-nodes  di  and  our  conditional  independence  (1),  we  cam  re¬ 
place  each  term  in  (2)  by  the  corresponding  conditional  probability,  i.e.,  for  the 
term  with  Ai  as  the  r.v.  being  conditioned  upon,  we  can  replace  it  with  the 
conditional  probability  associated  with  S-node  di.  Thus,  our  joint  probability 
P{Xr)  cam  be  computed  as  a  product  of  the  conditionals. 

Finally,  if  we  set  the  conditional  probability  for  d*  to  we  have 

i=l 


Applying  these  probabilistic  semantics  to  /C  is  the  basis  of  Bayesian  knowledge¬ 
bases  (bkbs).  Reasoning  over  BKBs  cam  thus  involve  seairch  for  the  most- 
probable  inference  in  R.  Carefully  selecting  the  inferences  which  make  up  R 
provides  a  wide  variety  of  different  reasoning  schemes.  For  example,  to  deter¬ 
mine  the  most  probable  state  of  the  world  when  given  certaun  evidence  cam  be 
achieved  as  follows:  Let  E  be  the  instantiation-set  representing  the  evidence. 
Our  goal  is  to  determine  the  complete  instamtiation-set  X*  which  maodmizes 

P{X*\E)^m^P{X\E) 


over  all  complete  instantiation-sets.  Note  that 


P{X\E)  =: 


PjX.E)  ^  P{X) 
P{E)  PiE)  * 


Since  P{E)  is  a  constant,  we  only  need  to  determine  the  most  probable  complete 
instantiation-set  X*  such  that  E  C  X* .  Thus,  we  can  accomplish  this  letting 
R  be  the  set  of  all  complete  inferences  that  also  contain  the  I-nodes  associated 
with  E. 

Performing  these  reasoning  computations  can  be  accomplished  in  a  wide 
variety  of  ways.  This  includes  basic  A*  seairch  and  integer  linear  programming 
[8]  for  exact  solutions  or  a  genetic  algorithms  approach  [2]. 


4  Conclusion 

By  carefully  defining  the  mathematical  model  and  formal  semantics  for  Bayesian 
knowledge-bases,  we  were  able  to  develop  a  new  fraunework  for  knowledge  rep¬ 
resentation  which  unifies  the  flexible  and  intuitive  structure  of  “if-then”  style 
rules  with  probability  theory.  The  new  knowledge  representation  is  fully  prob¬ 
abilistic  yet  also  retaiins  the  full  power  of  “if-then”  style.  Furthermore,  BKBs 
are  naturaJly  adept  at  handling  incompleteness  and  are  especially  well-suited 
for  knowledge  acquisition.  (For  more  details  on  using  BKBs  for  knowledge  ac¬ 
quisition,  see  [9].) 
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Clearly,  BKBs  can  subsume  Bayesian  networks  [7]  using  a  similar  probabilis¬ 
tic  semantics.  Unlike  Bayesian  networks,  BKBs  can  also  handle  various  forms  of 
cyclical  knowledge.  Finally,  it  seems  likely  that  temporal  information  coiild  also 
be  unified  into  BKBs  without  necessarily  resorting  to  time-slice  approaches.  In¬ 
stead,  time  can  be  used  as  a  variable  to  directly  influence  the  level  of  uncertainty 
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